Learning Imprecise Hidden Markov Models

Consider a stationary precise hidden Markov model (HMM) with n hidden states Xk, taking values xk in a set {1, . . . ,m} and n observations Ok, taking values ok. Both the marginal model pX1(x1), the emission models pOk|Xk(ok|xk) and the transition models pXk|Xk−1(xk|xk−1) are unknown. We can then use the Baum–Welch algorithm [see, e.g., 4] to get a maximum-likelihood estimate of these models. The Baum–Welch algorithm constructs the expected number of transitions ni j := ∑k=2 pXk−1,Xk|O1,...,On(i, j|o1, . . . ,on) from state i to state j in the whole Markov chain. If we do not have enough data to justify a precise probability model, such as the one we obtain using the classical Baum–Welch algorithm, then the approach we present is useful. Our contribution exists of a method for learning imprecise transition probabilities in an HMM. We are not aware of another such method in the literature. These transitions from a state Xk−1 = i to a state Xk = j are multinomial processes. The imprecise Dirichlet model (IDM) is a convenient model for describing uncertainty about such processes [3]. In order to learn using an IDM, we need the number of transitions and a choice for the pseudocounts s. Since the hidden states are unavailable, our method consists in taking the expected1 number of transitions (positive real numbers instead of natural numbers), derived from the Baum–Welch algorithm, rather than real counts. So, the lower probability for state j conditional on state i is estimated by Q({ j}|i) := ni j/(s+∑j=1 ni j). Learning an imprecise HMM, like our method does, is necessary before being able to make imprecise inferences. There are algorithms, like MePICTIr [2] and EstiHMM [1], to make exact inferences in an imprecise HMM. We applied our method to the following application. Given an observation sequence of counted earthquakes in a number of consecutive years, we are interested in predicting the number of earthquakes in the next years. The observed number of earthquakes in a year k is assumed to be generated by a Poisson process with rate λXk . Both the emission probabilities and the marginal probability are kept precise, while the transition probabilities are made imprecise, using our method. The following figure shows the credal sets for the transition model in the case m = 3, for different pseudocounts s. λ2 λ1 λ3 s = 1 s = 2 s = 3 s = 5 s = 10 s = 20 s = 50